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ABSTRACT 



The possibility of designing constrained adaptive finite 
impulse response digital filters is investigated as motivated 
by a study of adaptive noise cancellation. The first con- 
straint considered consists of a fixed angle between filter 
zeros and is implemented in a master-slave approach in which 
one of the zeros is adjusted adaptively and the others follow 
subject to the constraint. The second constraint considered 
is a linear constraint on the filter weights and is imple- 
mented by augmenting the error equation with Lagrangean 
multipliers. Simulations indicate that the approach is 
feasible . 
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INTRODUCTION 



I . 

This thesis investigates the possibility of designing 
constrained adaptive finite impulse response (FIR) filters. 
FIR adaptive filters self -ad just their parameters to match 
the output signal to a desired signal in an optimal least 
squares sense. Considered in detail is the possibility of 
designing the adaptive process so that either the parameters 
(weights) of the filter are constrained by an algebraic for- 
mula or, alternatively, are constrained so that the zero 
pattern of the FIR transfer function is fixed while the 
actual positions of the zeros are adaptively adjusted. An 
application which motivated this study is adaptive noise 
cancellation where noise rejection over a given bandwidth 
with a specified frequency rejection spectrum is required, 
but the center frequency of the rejection band is unknown and 
needs to be determined adaptively. 

The usual method of extracting a signal from a strong 
noise background is to pass it through a filter which tends 
to suppress the noise while leaving the signal relatively 
unchanged. Filters designed for this purpose can be either 
fixed or adaptive. The design of fixed filters is based on 
some prior knowledge of both the signal and noise character- 
istics. Adaptive filters have the distinct advantage of 
being able to adjust themselves automatically and their 
implementation requires little a priori knowledge of the 
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signal and noise characteristics. This type of adaptive 
filtering often converges to the optimal filter, which 
originated with the pioneering work of Wiener [Ref. 1] and 
later was enhanced by Kalman [Ref. 2] and others. The 
optimal Wiener filter is defined as the linear filter opti- 
mized with respect to a minimum mean squared error, where the 
error is the difference between the filter output and the 
desired filter output. 

Widrow [Ref. 3] presents the classic FIR adaptive filter 
which is optimized via a gradient minimum seeking algorithm 
called the Least Mean Squared (LMS) algorithm. This chapter 
discusses the theory behind both the Wiener and LMS filters, 
and demonstrates that, for statistically stationary input 
signals, the steady-state values of the LMS adaptive filter 
weights are accurate approximations of the Wiener weights . 

The following matrix, vector, and scalar definitions are 
used in this thesis. An underlined capital letter denotes a 
matrix (M) . A lower case underlined letter denotes a vector 
(v) . A lower case letter which is not underlined denotes a 
scalar (s) . Finally, a capital letter which is not under- 
lined denotes an internal element of the corresponding matrix 
M(n) . 

In Chapter II the concept of adaptive noise cancelling is 
studied in detail. In Chapter III the constrained adaptive 
FIR filter is presented and in Chapter IV some simulation 
results are given. 
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A literature search has indicated that very little re- 
search has been done in the area of constrained adaptive 
filters. Frost [Ref. 8] presents a constrained LMS algorithm 
which is capable of adjusting an array of sensors in real 
time to respond to a signal coming from a desired direction 
while discriminating against noises coming from other direc- 
tions. A set of linear constraints on the weights maintains 
a chosen frequency characteristic for the array in the 
direction of interest. 

A. THE ADAPTIVE LMS AND THE OPTIMAL WIENER FILTERS 

The LMS adaptive filter shown in Figure 1.1 uses the 
weighted sum of a set of input signals which are combined to 
form an output signal y (n) . The input signal vector is 
defined as 

X (n) = [x 1 (n) X 2 (n) 

= [X (n) X (n-1) . . . 

The weighting coefficients 

W = [W x W 2 ... W n ] T (1.2) 

are the weights of the system. Each input value is multi- 
plied by a corresponding weight coefficient and the linear 
combination of the sum of these weighted inputs forms the 
output 



X.. (n) ] 
N 



X (n-N) ] 



( 1 . 1 ) 
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INPUT SIGNALS 




Fig. 1.1. The LMS Adaptive Filter 



(1.3) 



N N 

y(n) l V'TX. (n) = l W.X(n-i) 

i=l i=l 

which can be written in matrix form as 

y (n) = X(n) T W = W T X(n) (1.4) 

An error signal is now defined as the difference between 
a desired response d(n) and the actual response y(n) . 

e (n) = d (n) - y (n) = d(n) - W T X(n) (1.5) 

The purpose of the adaptive algorithm is to adjust the 
weights of the filter to minimize the mean-square error. A 
general expression for mean-square error as a function of the 
weight values, assuming that the input signals and the de- 
sired response are statistically stationary, can be derived 
in the following manner. Squaring the error results in 

e (n) 2 = d (n) 2 - 2d (n) X (n) T W + W T X (n) X (n) T W (1.6) 

Taking the expected value of both sides yields 

E [ e (n) 2 ] = E[d(n) 2 ] - 2E [d (n) X (n) T ] W 

+ W T E [X(n) X (n) T ] (1.7) 

If the vector r , is now defined as the cross correlation 
— xd 

between the desired response (a scalar value) and the input 
vector, the result is 
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= E[d(n)X(n)] = E[d(n)X(n) d(n)X(n-l) ... d(n)X(n-N)] (1.8) 



The input correlation matrix R is defined as 

XX 



-xx = E ^(n)X(nr] = E 



Xl (n)Xl (n) Xl(n)X2(n) ... Xl(n)XN(n) 
X2(n)Xl(n) X2(n)X2(n) ... X2(n)XN(n) 



= E 



XN(n)Xl(n) XN(n)X2(n) ... XN(n)XN(n) 



(1.9) 



X(n) 2 X(n)X(n-l) ... X(n)X(n-N)' _ l 

X(n-l)X(n) X(n-l) 2 ... X(n-l)X(n-N) 



X(n-N)X(n) X(n-N)X(N-1) ... X(n-N)' 



( 1 . 10 ) 



R 

—xx 



10) 

«* 


is 


usually written 


as : 




R 


(0) 


R (1) 


R 


(N) 


XX 


XX 


XX 


R xx 


(1) 


R (0) 

XX 


R 

XX 


(N-l) 



R (N) R (N-l) 

XX XX 



R (0) 
xx 



( 1 . 11 ) 



Now the mean squared error (1.7) can be expressed as 
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E [ £ (n) 2 ] = E [d (n) 2 ] - 2 £ xd ; I + ^R^W (1.12) 

Since the error is a quadratic function of the weights 
and R v is a positive definite matrix then the surface of the 
error has a guaranteed global minimum. Gradient methods 
adjust the weights to minimize the error by descending along 
this surface with the objective of finding the bottom. 

The gradient V_ of the quadratic error function is obtained 
by differentiating Equation (1.12) with respect to the weight 
vector w. 
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{ 



3E [ e (n j ] 
9w. 



3E [ s (n) j 
3w„ 



3E [ e (n ; ] 



3w. 



N 



T 

} 



(1.13) 



V 




+ 2R w 

—XX— 



(1.14) 



This optimal weight vector w is called the Wiener weight 
vector [Ref. 1] and is found by setting the gradient of the 
mean square error function to zero. 



W* 




-xd 



(1.15) 



This equation is a matrix version of the Wiener-Hopf 
equation. The practical objective of the adaptive system is 
to find a solution to (1.15). An exact solution would require 
a priori knowledge of the correlation matrices r ^ and R xx * 
However, this information is usually not available. Addi- 
tionally, when the number of weights is large a direct 
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solution is computationally cumbersome since it requires an 
N by N matrix inversion in addition to the N(n+l)/2 auto- 
correlation and cross correlation measurements . 

B. AN EXAMPLE OF THE WIENER SOLUTION 

As an example to illustrate the calculations involved 
for a simple four weight Wiener solution consider the 
following example using deterministic signals. The input 
signal is the sampled sum of two sinusoids of different 
frequencies and is given by 

X(nT) = sin(ujjnT) + sinfu^nT) (1.16) 

The desired value is the sampled desired or reference 
signal. 



d(nT) = K sinfc^nT) (1.17) 

In order to calculate the autocorrelation matrix, R , note 

XX 

that 



XI (n) 


= X (n) = 


sin(co^nT) + sinfu^nT) 


X2 (n) 


= X(n-l) 


= sin (oj^ (n-1) T) + sin ( (n-1) T) 


X3 (n) 


= X (n-2 ) 


= sin (w^ (n-2) T) + s'. in (u^ (n-2) T) 


X4 (n) 


= X (n-3 ) 


= sin (uj ^ (n-3 ) T) + s in (u^ (n-3) T) 
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The following expected value computations are taken over a 
full number of cycles, P, for both sinusoids. 



E [XI (n) XI (n) ] = E[Xl(n) 2 ] = XI (n) 2 



= I + I = 1 
2 2 



Likewise all the diagonal terms will take on the value of 



X2 (n) 2 = X3 (n) 2 = X4 (n) 2 = 1 



Consider now the off diagonal terms of R 

XA 



E [XI (n)X2 (n) ] * Xl(n)X2(n) 

P 



l (sin u^nT+sin co 2 nT) (sinw^ (n-1) T + sinco^ (n-1) T) 
n=0 



Carrying out the indicated multiplication yields 



P 

E [XI (n) X2 (n) ] = £ (sinoj.nT) (sinw. (n-l)T) 

n=0 1 

P 

+ l (sino) 2 nT) (sino) 2 (n-1) T) 
n=0 

P 

+ l (sinu.nT) (sinw^j (n-1) T) 
n=0 1 * 

P 



+ \ (sinu_nT) (sincj, (n-1) T) 

n=0 ^ 
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The final two terms of this expression are zero leaving 



E [XI (n) X2 (n) ] = £ (sinc^nTsin ( w^nT-co^T) +sinw 2 nTsin (c^nT-u^T) ) 



n=0 



Using the identity (sinA) (sinB) = i[cos (A-B) -cos (A+B) ] the 
final expression becomes 



E [XI (n)X2 (n) ] = 



j coscu^T + j cosu^T 



(1.19) 



Similarly, 



E [XI (n) X3 (n) ] = 



2 " cos2co^T 



j cos2to 2 T 



( 1 . 20 ) 



E [XI (n) X4 (n) ] = 



j cos3cu^T 



j cos3u) 2 T 



( 1 . 21 ) 



Denoting 



E [XI (n)X2 (n) ] = a 



E [XI (n) X3 (n) ] = b 



( 1 . 22 ) 



E [XI (n) X4 (n) ] = c 



the autocorrelation matrix can be expressed as 



*xx 



1 

a 

b 

c 



a 

1 

a 

b 



b 

a 

1 

a 



c 

b 

a 



(1.23) 
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For the cross correlation matrix r 



£ xd = E [d (n) XI (n) d(n)X2(n) d(n)X3(n) d(n)X4(n)] r (1.24) 



These terms compute as 



E[d(n)Xl(n)] = K (sinu^nT + sinu^nT) sino^nT 



P 

= K l sinw^nT (sinw^nT + sinw 2 nT) 
n=0 



p 2 P 

= K l sin ni,nT + K £ (sinu, nT) sincu 9 nT 
n=0 1 n=0 1 



The final term sums to zero leaving 



E [d (n)Xl (n) ] = K [ sin^nT = | (1.25) 

n=0 1 



Similarly, 



E[d(n)X2(n)] = K (sinw^ (n-1) T + sina^ (n-1) T) sinco^nT 



P 

= K l sinoj-^nT (sinu^ (n-1) T) 
n=0 

P 

+ K l sincu, nT (sinu>„ (n-1) T) 
n=0 



The final term sums to zero leaving 
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E[d(n)X2 (n) ] 



p 2 P 

= K[ l sin u,nT • j sinto, nTsinw, T] 
n=0 1 n=0 1 1 



Using the (sinA) (sinB) trigonometric identity invoked for 
(1.19) and realizing that the final term sums to zero, the 
expected value finally becomes 



E[d(n)X2(n)] = j cosoj^T 

Likewise, 

E[d(n)X3(n)] = j cos2u)^T 



and 



(1.26) 



(1.27) 



E[d(n)X4(n) ] 



| cos34^T 



(1.28) 



The Wiener solution for w* as given by (1.15) is 







— — 


-1 




1 a b c 




1 


a 1 a b 




COSto^T 


b a 1 a 




cos2uj^T 


c b a 1 




cos3oj, T 


— ^ 







(1.29) 



where the values of a, b and c are given by (1.22). The 
Wiener vector is calculated for the specific values of 



K = 1 

fl = 10 Hz 
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f 2 = 35 Hz 



T = 1/256 

For these specific values 
a = .8116 
b = .3676 
c = -.052 
^•[coscOjT] = .485 



y[coso),T] = .441 



jtcosajjT] = .3705 

and the optimal Wiener vector w* is 



w 



* = 









w* 




3.0614 


W 2 




-5.5772 


w* 




5.1228 


w 4 




-1.5780 


— 


i 





(1.31) 



C. THE LMS ADAPTIVE FIR FILTER 

The values for the correlation matrices are not generally 
known a priori. The LMS adaptive algorithm introduced by 
Widrow and Hoff [Ref. 1] is a practical method for finding 
close approximate solutions to (1.15) in real time. The 
algorithm does not require measurements of correlation 
functions, nor does it require matrix inversion. The LMS 
algorithm is an implementation of the method of steepest 
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descent. According to this method, the "next" weight w(n+l) 
is equal to the present weight w(n) plus a change proportional 
to the negative gradient . Thus 

w(n+l) = w(n) - mV (n) 



The parameter m controls stability and also the rate of 
convergence. An estimate of the instantaneous gradient V (n) 
is obtained by assuming that the square of a single error 
sample s(n) is an estimate of the mean square error. Differ- 
entiating e (n) with respect to w results in 



V (n) 



V(n) 



, 9s (n) ^ 9 e (n ) ^ 9 £ (n) ^ i T 

9w. 9w» * 9w 

12 n 

T -/_w3e(n) 3e(n) Ssln), 11 

2 £<n) [ -^t— szr- ••• -9vr J 

12 N 



(1.32) 



(1.33) 



The expression for the gradient estimate can be approximated 
by 



V(n) = - 2£(n)X(n) (1.34) 

Using this estimate in place of the true gradient yields the 
Widrow-Hoff LMS algorithm given by 

w(n+l) = w(n) + 2ye(n)X(n) (1.35) 

The algorithm is generally easy to implement and al- 
though it makes use of gradients of the mean square error 
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function, it does not require squaring, averaging or 
differentiation . 

To show the convergence of the Widrow-Hoff LMS algorithm 
(1.35) to the Wiener solution given by (1.15) write 



w (n+1) 


= w (n) + 2y e (n) X (n) 


as 




w (n+1) 


= w(n) + 2yX(n)[d(n) -X(n) T w(n)] 


w (n+1) 


= [I - 2yX(n)X (n) T ]w(n) + 2yX(n)d(n) 


Consider now the 


ensemble average. That is. 


E [w (n+1) ] = 


[I -2yR xx ]E[w(n) ] + 2yr xd 



With an initial weight vector w(0) , j+1 iterations of 
Equation (1.36) becomes 

E [w ( j+1) ] = [I-2yR xx ] j+1 w(0) +2y ^[I-2yR xx ] 1 r xd (1.37) 

This equation may be put in diagonal form by using the normal 
form expansion of the matrix R ; that is 

XX 

—xx • °~ 1a, 2 
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where A is the diagonal matrix of eigenvalues, and the square 
matrix of eigenvectors is the matrix Q. Equation (1.37) 
can now be written as 



[ [w ( j+1) ] = [I-2uQ' 1 AQ] j+1 w(0) +2y I [I-2yQ _1 AQ] 1 r , 

i=0 - 

= Q" 1 [I-2yA] j+1 Qw(0) + 2y j [I-2yQ _1 AQ] A r , (1.39) 

i=0 - 

As long as the terms of the diagonal matrix [I-2yA] are all 
of magnitude less than unity, then the first term of (1.39) 
goes to zero as the number of iterations increases. That is, 

lim[I-2yA] j+1 - 0 (1.40) 

j ->oo 



Writing (1.40) as a geometric series results in 



lim j [I-2y A] 1 = A _1 (1.41) 

j+oo i=o y 



or for the specific component 'p' of the matrix 



lim l [I-2y A] 
i=o 



2y A 



Therefore, in the limit, Equation (1.39) becomes 



lim E [w ( j+1) ] = Q 1 AQ£ xd = R xx £ xd (1.42) 

j ->CO 
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which is the same as the Wiener solution shown in Equation 
(1.15) . 



Convergence of the mean of the weight vector to the 
Wiener solution is insured if and only if the proportionality 
constant y is set within certain bounds . Since the diagonal 
terms of [I-2yA] must all have magnitude less than unity, 
and since all eigenvalues in A are positive, the bounds on y 
are given by 




or 



0 < y < 1/A 



max 



(1.43) 



where A 



max 



is the maximum eigenvalue of R . 

""XX 
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II. ADAPTIVE NOISE CANCELLING AND ITS APPLICATIONS 



In this chapter the concept of adaptive noise cancella- 
tion is considered. Adaptive noise cancelling is one of the 
most practical applications of adaptive signal processing 
[Refs. 4, 5, 6, 7]. The basic principle involved is the use of 
a reference input derived from one or more sensors located 
at points in the noise field where the signal is either unde- 
tectable or weak. The reference input is adaptively filtered 
and subtracted from the primary input containing both signal 
and noise to generate an error signal which controls the 
adaptive process. The result is the attenuation or elimina- 
tion of the primary noise by cancellation. In circumstances 
where adaptive noise cancelling is applicable levels of noise 
rejection are often attainable that would be difficult or 
impossible to achieve through direct filtering. Because the 
concepts of adaptive noise cancelling and their applications 
are fundamental to constrained adaptive filtering developed 
in Chapter III, they are presented here in detail. For exam- 
ple, if the angle between zeros and the magnitude of the zeros 
are maintained constant during an adaptive process, the fre- 
quency response characteristics remains invariant. Specific 
examples are discussed and the resulting equations indicate 
the design limitations of these approaches . 

A. THE BASIC SYSTEM 

Figure 2.1 depicts the basic adaptive noise cancelling 
system concept . A signal [s] is transmitted over a channel 
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1 




Fig. 2.1. The Adaptive Noise Cancelling Concept 
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to a sensor that picks up a noise [nO] which us uncorrelated 
with the signal. The combined signal and noise [s+nO] form 
the primary input to the canceller. A second sensor re- 
ceives a noise [nl] which is also uncorrelated with the 
signal but which is correlated in some unknown manner with 
the noise [nO] . This sensor input provides the reference 
input to the canceller. The noise [nl] is filtered to pro- 
duce an output [y] that is as close a replica as possible 
to [nO] . This output is subtracted from the primary input 
[s+nO] to produce the system output [z = s +n0 -y] . 

If it were possible to know the characteristics of the 
channels over which the noise was transmitted to the primary 
and reference sensors, then it would theoretically be possi- 
ble to design a fixed filter capable of changing [nl] into 
[nO] . The filter output could then be merely subtracted from 
the primary input to produce the signal alone. However, the 
characteristics of the transmission paths are generally un- 
known or known only approximately and are seldom of a fixed 
nature, therefore precluding the use of a fixed filter. 

In the system shown in Figure 2.1 the reference input is 
processed by an adaptive filter. Self -adjustment of the 
filter's impulse response is accomplished by the LMS algorithm 
described in Chapter I. The error signal used in the adaptive 
process depends on the nature of the application. With noise 
cancelling systems the practical objective is to produce a 
system output [z = s +n0 -y] that is a best fit in the least 
squares sense to the signal [s] . 
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Consider the system inputs shown in Figure 2.1 and the 
filter output [y] . Assume that [s] is uncorrelated with [nO] 
and [nl] and that [nO] and [nl] are correlated. The output 
[z] is 



z = s + nO - y (2.1) 

Squaring produces 

z 2 = s 2 + (nO-y) 2 + 2s(n0-y) (2.2) 

Now, taking the expectation of both sides and using the fact 
that [s] is uncorrelated with [nO] and [y] produces 

E[z 2 ] = E[s 2 ] + E[(n0-y) 2 ] + 2E [s (nO-y) ] 

= E[s 2 ] + E [ (nO-y) 2 ] (2.3) 

The minimum output power is 

Min E[z 2 ] = E[s 2 ] + Min E[(n0-y) 2 ] (2.4) 

The signal power is unaffected as the filter is adjusted to 

2 2 
minimize E[z ]. When the filter is adjusted so that E[z ] 

2 

is minimized, E [ (nO-y) ] is therefore also minimized. The 
filter output [y] is then a best least squares estimate of the 
primary noise [nO] . It is also of interest to note that 

2 2 

when E[(nO-y) ] is minimized, E[(z-s) ] is also minimized, 
since from (2.1) 
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(z-s) 



(nO-y) 



(2.5) 



Adapting the filter to minimize the total output power causes 

the output [z] to be a best least squares estimate of the 

signal [s] for the given reference input. The output [z] 

will contain the signal [s] plus noise. From (2.1) the output 

2 

noise is given by (nO-y) . Since minimizing E[z ] minimizes 
2 

E[(nO-y) ], minimizing the total output power minimizes the 
output noise power. Because the output signal remains con- 
stant, minimizing the total output power maximizes the signal- 

to-noise ratio. From (2.3) it can be seen that the smallest 

2 2 

possible output power E[z ] = E[s ] is achieved when 
2 

E[(nO-y) ] = 0 and therefore y = nO and z = s. In this case, 
minimizing output power causes the output signal to be per- 
fectly noise free. These same arguments can be extended to 
the case where the primary and reference inputs contain, in 
addition to [nO] and [nl] , additive random noises uncorre- 
lated with each other and with [s] , [nO] and [nl] . They can 
also be extended to the case where [nO] and [nl] are deter- 
ministic rather than stochastic. 

B. SIGNAL-TO-NOISE RATIO IN THE ADAPTIVE FILTER 

At this point it is of value to show analytically the 
increase in signal-to-noise ratio of the noise cancelling 
technique . 

As noted previously, fixed filters are generally inappro- 
priate for noise cancelling situations because the correlation 
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and cross correlation functions are usually unknown and 
additionally they often vary with time. Adaptive filters 



"learn" these statistics initially and then track them through 
slow variations. For stationary stochastic inputs, however, 
the steady-state performance of adaptive filters closely 
approximates that of fixed Wiener filters and therefore Wiener 
filter theory provides a useful mathematical tool in care- 
fully analyzing statistical noise cancelling problems. 

Figure 2.2 shows the classic single-input/single-output 
Wiener filter. The input signal is x(j), the output signal 
is y(j) and the desired response is d(j). The input and 
output signals are assumed to be discrete in time, and the 
input signal and desired response are assumed to be statis- 
tically stationary. The error signal is e(j) = d(j) - y(j). 
The filter is linear, discrete and designed to be optimal 
in the minimum mean-square-error sense. It is considered 
to be composed of an infinitely long, two sided tapped delay 
line . 

As shown in Chapter I (1.15) the optimal Wiener weight 
solution can be written 



w* (k) 




( 2 . 6 ) 



or 



oo 






(2.7) 
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Fig. 2.2. The Wiener Filter 
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To obtain the transfer function of the Wiener filter 
consider first the power spectral density of the process. 
The application of the z-transform to R (k) yields 

X X 

oo 

S xx (z) £ z[R xx (k)] = J R xx (k)z" k (2.8) 

Likewise, the cross power spectrum between the input signal 
and desired response is 

S xd (z) - 2[ £xd (k>1 - , I -xd (k> z " k (2 - 9) 

— k=-°° — 

The transfer function of the Wiener filter is 



w * ( z ) = l w*(k)z _k (2.10) 

k=-°° 

For specific values of an individual matrix the optimal 
Wiener transfer function can be written as 



w* (z) 



S xd (z > 



( 2 . 11 ) 



Consider now a single channel adaptive noise canceller 
with a typical set of inputs shown in Figure 2.3. The pri- 
mary input consists of a signal s(j) plus the sum of two 
noises mO(j) and n(j). The reference input consists of a 
sum of two other noises ml(j) and n(j) * h(j), where h(j) is 
the impule response of the reference channel whose transfer 
function is H(z). To simplify the notation the transfer 
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Fig. 2.3. Adaptive Noise Canceller with Correlated and 
Uncorrelated Noises in the Primary and 
Reference Inputs 
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function of the noise path from n(j) to the primary input 
has been set at unity. This procedure does not restrict the 
analysis since a suitable choice of H(z) and of statistics 
for n(j) will allow any combination of mutually correlated 
noises to appear at the primary and reference inputs. The 
noises mO(j) and ml(j) are uncorrelated with each other, with 
s(j) and with n(j) and n(j) * h(j) . The noises n(j) and 
n(j) * h(j) have a common origin and are correlated with each 
other but uncorrelated with s(j). 

The noise canceller includes an adaptive filter whose 
reference input x(j) isml(j) +n(j) *h(j) and whose desired 

response d(j) is the primary input to the noise canceller and 
is composed of s(j) + mO(j) + n(j). The error signal e(j) 
is the noise canceller's output. When the adaptive process 
has converged then the optimal unconstrained transfer function 
of the adaptive filter is given by (2.11), which can be further 
reduced as follows. The spectrum of the noise ml is 
S mlml^ an< ^ that of the n °i se n arriving via H(z) is 
S nn (z) | H ( z ) | . Therefore, the input spectrum to the filter 

is 

S (z) = S, ml (z) + S (z) |H (z) | 2 (2.12) 

xx mlml nn 1 

The cross power spectrum between the filter's input and 
the desired response depends only on the mutually correlated 
primary and reference inputs and is given by 
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(2.13) 



S ,(z) = S (z)H(z -1 ) 

xd nn 

Substituting Equations (2.12) and (2.13) into (2.11) results 
in the Wiener transfer function and is given by 

S^ n (z)H(z -1 ) 

w*(z) = — j (2.14) 

S . . (z) +S (z) I H ( z) I 

mlml nn 1 1 



The result is a transfer function which is independent of 

the primary signal spectrum S (z) and of the primary uncorre- 

s s 

lated noise spectrum s m Q m Q( z )* 

An interesting special case which clearly brings out the 
function of the adaptive noise canceller is when the additive 
noise ml in the reference input is zero. Then s m j_ m ]_( z ) is 
zero and the optimal transfer function becomes 



w*(z) = 1/H(z) 



(2.15) 



That is, the adaptive noise canceller causes the n(j) noise 
to be perfectly nulled at the noise canceller output. The 
primary uncorrelated noise mO(j) remains uncancelled. 

Consider an analysis of the performance of the adaptive 
noise canceller in terms of the ratio of the signal power 
density at the output, P ou t-( z )' to t * ie n °i se power density at 
the primary input p . (z) . This ratio can be written as 



p out (2) 

P • (z) 
pri 



S (z) + S A n (z) 
nn mOmO 

S ( z ) 

output noise 



primary noise 
pwr spectrum 
output noise 
pwr spectrum 



(2.16) 
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with the signal power spectrum factored and cancelled out of 
the numerator. Figure 2.3 shows that the output noise 
spectrum consists of the sum of three components — one due to 
the propagation of mO(j) directly to the output, another due 
to the propagation of ml(j) to the output via the transfer 
function W(z), and another due to the propagation of n(j) 
to the output via the transfer function 1 -H(z)W(z). The 
output noise power spectrum can then be written as 



^output noise ^ S m0m0^ z ^ +S mlml^ z ^ ^ z ^ 



+ S(z) |1 -H(z)W*(z) 
nn 1 



(2.17) 



Now, if the ratios of the spectra of the uncorrelated noises 
to the spectra of the correlated noises (noise-to-noise den- 
sity ratios) at the primary and reference inputs are defined 
as 



A(z) 



B(z) 



S mOmO (z> 
S nn (z) 



S mlml (z> 



(z) H ( z ) 
nn 



(2.18) 



(2.19) 



then the transfer function of (2.14) can be written as 



W*(z) = l/[H(z) (B(z)+1) ] (2.20) 

and the output noise power spectrum (2.17) can be written 
as 
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S ( z ) 

output noise 



S 




mOmO 



H ( z ) | 2 (B(z)+1) 





( 2 . 21 ) 



and the ratio of the output to the primary input noise power 
spectra given in (2.16) is 



S ( z ) 

output noise v 



[1 + A ( z) ] 

A(z) + (B (z) ) /(B (z)+l) 



= [A ( z) + 1] (B (z) + 1] 
A(z) + A(z)B(z) + B(z) 



This expression allows an estimation of the level of noise 
reduction to be expected with an ideal noise cancelling sys- 
tem. In such a system the signal propagates to the output 
with a transfer function of unity. From (2.22) it can be 
seen that the ability of a noise cancelling system to reduce 
noise is limited by the uncorrelated-to-correlated noise den- 
sity ratios at the primary and reference inputs. The smaller 
in magnitude are A(z) and B(z), the greater will be 

P . (z)/p . (z) and the more effective the action of the 

out ' pri 

canceller. The desirability of low levels of uncorrelated 
noise in both inputs is made even more evident 




S (z) [1 + A (z) ] 
nn 
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by considering the approximations 



1) small A(z) 




1 + B ( z) 
B (z) 



(2.23) 



2) small B (z) 




1 + A(z) 
A ( z) 



(2.24) 



3) small A ( z ) 
and B ( z ) 




A (z ) + B ( z) 



1 



(2.25) 



Infinite improvement is implied by these relationships 
when both A(z) and B(z) are zero resulting in complete re- 
moval of noise at the system output and perfect signal repro- 
duction. When both A(z) and B(z) are small other factors 
such as mis adjustment caused by gradient estimation noise in 
the adaptive process and the finite length of the adaptive 
filter limit system performance. These factors are discussed 
at some length in [Ref. 3] . 

C. SIGNAL PROPAGATION IN THE REFERENCE INPUT 

If it is reasonable to consider reference noise propaga- 
tion into the primary signal input it is also reasonable to 
consider certain instances when the signal propagates to the 
reference input. The system depicting the adaptive noise 
canceller with signal components in the reference input is 
shown in Figure 2.4. The derivation which discusses how much 
of the signal is cancelled when a portion of the signal input 
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Fig. 2.4. Adaptive Noise Canceller With Signal 
Components in the Reference Input 
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"leaks" into the reference input omits the additive uncorre- 
lated noises mO(j) and ml(j) in order to simplify the 
analysis . 

Given that the spectrum of the signal is S (z) and 

s s 

that of the noise is S (z) , then the spectrum of the 

nn 

reference input is given by 

S xx (z) = S ss (z) | X (z) | 2 + S nn (z) |H(z) | 2 (2.26) 

The cross spectrum between the reference and primary inputs 
is 



S xd (z) 



- S (z)I(z' 1 ) + S (z)H(z' 1 ) 
ss nn 



(2.27) 



When the adaptive process has converged, the Wiener trans- 
fer function of the adaptive filter given by (2.11) is 



W* (z) 



S (z)I(z" 1 ) + S (z)H(z 1 ) 
ss nn 

S oc (z) 1 1 ( z ) I 2 + S nri (z) | H ( z) I 2 
s s 1 1 nn 1 1 



(2.28) 



The transfer function of the propagation path from the 
signal input to the noise canceller output is l-I(z)W*(z) 
and that of the path from the noise input to the canceller 
output is l-H(z)W*(z). The spectrum of the signal component 
in the output is thus 



> . = S (z) |l - I(z)W*(z) 

ss out ss 1 



= S (z) 
ss 



- 1 , 



| H( z) -I (z) |S nn (z)H(z -J -) 



S (z) |l(z) | 2 +S o (z) | H ( z) | 2 
ss ' ' nn 



(2.29) 
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and likewise, that of the noise is 



S .. x. ( z ) 

n in out* 



S (z) I 1 - H (z)W* (z) 
nn 1 



S (z) 
nn 



tI(z)-H(z) ] S (z) I (z 1 ) 

b b 



S ss (z) 1 1 (z) | 2 +S nn (z) (H ( z) ) 2 



( 2 . 30 ) 



The output signal-to-noise density ratio is therefore 



»out (z) 



S ss (z) 

S nn (z) 



S nn (z) H(z ' 1) 



S (z) I (z _1 ) 

S 5 



S nn (z) I H ( z) 
S ss (Z > I 1 (Z) 



( 2 . 31 ) 



The output signal-to-noise density ratio can be conveniently 
expressed in terms of the signal-to-noise density ratio at 
the reference input in the following manner. The spectrum 
of the signal component in the reference input is 



S x- 
ss ref 



S ss (z) |I(Z> I 2 



( 2 . 32 ) 



and that of the noise component is likewise 



S 



nn ref 



(z) 



S 

nn 



(z) | H (z) 



2 



( 2 . 33 ) 



Therefore, the signal-to-noise density ratio at the reference 
input is thus 
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(2.34) 



S ss (z) 1 1 (z) l 2 

PrSf s nn (z) l H(z) I 2 

Comparison of Equation (2.34) with (2.31) shows that 

p , ( z ) = 1/p ,(z) 

out ref 

This shows that if the noises in the primary and reference 
inputs are mutually correlated, the signal-to-noise density 
ratio at the noise canceller output is simply the reciprocal 
at all frequencies of the signal-to-noise ratio at the reference 
input. That is, in order to obtain a good signal-to-noise 
density ratio at the filter output there should be very little 
signal at the reference input. 

The final objective of the analysis is to derive an 
expression for the spectrum of the output noise. As with the 
previous analysis it is instructive to first write the trans- 
fer function for the path from which the noise n(j) propagates 
to the output. 



1 - H ( z) W* ( z) 



S (z) I (z -1 )+S (z) H (z _1 ) 2 

1 — h ( z ) ss 

S ss (z) 1 1 (z) | 2 +S nn (z) | H (z) | 2 



S (z) I (z" 1 ) [I (z) -H (z) ] 
ss 

S (z) |I (z) I 2 +S (z) | H (z) I 2 

ss 1 nn 



(2.36) 



When I (z) is small (2.36) reduces to 
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1 - H(z)W*(z) 



(2.37) 



~S (z) I ( z - 1 ) 
ss 

S (z)H(z” 1 ) 
nn 

The output noise spectrum is 



S 



output noise 



2 

S nn (z) |1 - H(z)W* (z) | 



(2.38) 



Again, considering the case where I(z) is small results in 



S output noise ^ S nn^ z ^ 



s ss u)uz-h 



S (z)H(z" 1 ) 
nn 



(2.39) 



If Equation (2.39) is written in terms of the signal-to-noise 
density ratios at the reference and primary inputs, where 
the signal-to-noise density ratio at the reference input is 
given by Equation (2.34) and 



a s (z) 
„ , _ \ A ss 

pri S ( z ) 

nn 



(2.40) 



then Equation (2.39) can be written as 



’output noise 



S ( z) I p j. (z) 
nn 1 K ref 



P • (z) 
pri 



(2.41) 



Equation (2.41) shows that the output noise spectrum 
acts according to three factors (given that I(z) is small). 
First, the output noise spectrum depends on the input noise 
spectrum. Second, if the signal-to-noise density ratio at 
the reference input is low, the output noise will be low; 
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that is, the smaller the signal component feeding into the 
reference input, the better the cancellation of the noise. 

This is to be expected and was already shown by Equation 
(2.35). The third factor implies that if the signal-to-noise 
density ratio in the primary input (the desired response of 
the adaptive filter) is low, the filter will be trained most 
effectively to cancel the noise rather than the signal and 
therefore the output noise will be low. 

As an illustration of the level of performance attainable 

in practical situations consider the following example. An 

adaptive noise cancelling system is designed to pass a 

plane-wave signal received in the main beam of an antenna 

array and to reject strong interference in the near field or 

in a minor lobe of the array. Assume that the signal and 

interference power spectra are overlapping and that the 

interference power density is twenty times greater than the 

signal power density at the individual array elements. Then 

the signal-to-noise ratio at the reference input P re £ is 

1/20. Assume also that because of array gain the signal power 

equals the interference power at the array output which forms 

the primary input to the adaptive system. The signal-to-noise 

ratio at the primary input is p . = 1. After convergence 

pr x 

the signal-to-noise ratio at the system output will be 



p out ^ p ref 




20 



If signal distortion is defined as 
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D ( z) 




(2.42) 



then the maximum signal distortion will be 



D(z) 



(1/20) /I = 5 percent 



The adaptive cancelling improves the signal-to-noise 
ratio twentyfold while introducing only 5 percent distortion. 
Additionally, the adaptive filter will provide the same per- 
formance when the input conditions change and a new set of 
convergent weights have been obtained. 

D. NOISE CANCELLING APPLICATIONS 

This section describes several practical applications 
which demonstrate the applied concepts of adaptive noise 
cancelling. These applications include cancellation of noise 
in speech signals, cancellation of antenna sidelobe inter- 
ference, cancellation of 60-Hz interference and cancellation 
of either periodic or broadband interference when no reference 
is available. 

A prime example of noise contaminated speech is that of a 
pilot communicating by radio from the cockpit of an aircraft 
where a high level of engine noise is interfering with the 
pilot's voice. The noise contains, among other components, 
strong periodic mixtures that occupy the same frequency band 
as speech. These components cannot be "low filtered" or 
"high filtered" out of the speech pattern and are picked up by 
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the microphone into which the pilot speaks, severely inter- 
fering with the intelligibility of the radio transmission. 

It is impractical to process the transmission with a conven- 
tional fixed filter because the frequency and intensity of 
the noise components vary with engine speed and load and even 
the position of the pilot's microphone. By placing a second 
microphone at a suitable location in the cockpit, a sample 
of the ambient noise which is free of the pilots speech can 
be obtained. This sample can be filtered and subtracted from 
the transmission, significantly reducing the interference. 

Widrow et al . , [Ref. 4] demonstrated the feasibility of 
cancelling noise in speech signals by conducting a number of 
experiments which simulated the cockpit noise problem. Figure 
2.5 shows the system used for cancelling the noise in the 
cockpit noise simulations . A voice input from a room with 
strong acoustic interference was used as the primary input. 

A second microphone was placed in the room away from the 
speaker and this was used as the reference input. The output 
of the noise cancelling was then monitored by a remote listener. 
The canceller included an adaptive filter with 16 weights 
whose values were digitally controlled by a computer. A 
typical experiment used an audio frequency triangular wave 
containing many harmonics as interference. Because of multi- 
path effects the amplitude and phase of the interference 
varied from point to point in the room. The noise cancelling 
system was able to reduce the output power of the interference, 
which otherwise made the speech unintelligible, by 20 to 25 
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ADAPTIVE NOISE CANCELLER 



Fig. 2.5. Cancelling Noise in Speech Signals 
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dB, rendering the interference barely perceptible to the 
remote listener. No noticeable distortion was introduced 
into the speech pattern. Convergence times were on the order 
of a few seconds and the system was readily able to readapt 
when the position of one or both microphones was changed or 
when the frequency of the interference was varied over the 
range of 100 to 2000 Hz. 

E. CANCELLING ANTENNA SIDELOBE INTERFERENCE 

Another type of noise cancelling is that of eliminating 
strong unwanted signals which are incident on the sidelobes 
of an antenna array. These interferences can severely retard 
the reception of weaker signals on the main beam. The 
conventional method of reducing this type of interference by 
adaptive beamforming is often complex and expensive to imple- 
ment. When the number of spatially discrete interference 
sources is small, adaptive noise cancelling can provide a 
simpler and less expensive method of coping with this problem. 
Consider an array pattern with signal strengths and directions 
as shown in Figure 2.6. The array consists of a circular 
pattern of 16 equally spaced omnidirectional elements. The 
outputs of the elements are delayed and summed to form a main 
beam steered at a relative angle of 0 degrees. A simulated 
"white" signal consisting of uncorrelated samples of unit 
power is assumed to be incident on the beam. Simulated inter- 
ference with the same bandwidth and with a power of 100 is 
incident on the main beam at a realtive angle of 58 degrees. 
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Fig. 2.6. Array Configuration for Adaptive 
Sidelobe Canceller 
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The entire array is then connected to the adaptive noise 
cancelling system shown in Figure 2.7. In this case the 
output of the beamformer serves as the canceller's primary 
input, and the output of one of the elements (#4) is arbi- 
trarily chosen as the reference input. The adaptive can- 
celler uses 14 weights. 

A number of experiments performed in [Ref. 4] show that 
the signal-to-noise ratio at the system output was found 
after convergence to be +20 dB. The signal-to-noise ratio 
at the single array element was -20 dB. This result bears out 
the equation shown in the Wiener solution (2.35), that the 
signal-to-noise ratio at the system output would be the 
reciprocal of the ratio at the reference input, which is 
derived from a single element. 

F. CANCELLING 60-HZ INTERFERENCE IN ELECTROCARDIOGRAPHY 

A practical example of cancelling 60-Hz interference is 
found in electrocardiography. A major problem which exists 
in the recording of electrocardiograms (ECG's) is the ap- 
pearance of unwanted 60-Hz interference in the output. Vari- 
ous methods have been utilized to help cancel the 60-Hz 
interference, including more effective grounding techniques 
and the use of twisted cabling. Another method capable of 
reducing 60-Hz ECG interference is adaptive noise cancelling 
via a system such as that shown in Figure 2.8. 

The primary input is taken from the ECG preamplifier and 
the 60-Hz reference is taken from a wall outlet. The adaptive 
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Fig. 2.7. Adaptive Noise Cancelling Applied to a 
Receiving Array 
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Fig. 2.8. Cancelling 60-Hz Interference in Electro- 
cardiography 
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filter is simple, containing only two variable weights, one 
applied to the reference input directly and the other to a 
version of it shifted in phase by 90 degrees. The two 
weighted versions of the reference are summed to form the 
filter's output, which is then subtracted from the primary 
input. A valuable advantage in the use of an adaptive filter 
rather than a fixed notch filter at 60-Hz, is that the varia- 
ble weights allow the 60-Hz interference to change in both 
magnitude and phase and still realize effective cancellation. 

G. CANCELLING PERIODIC INTERFERENCE WITH NO EXTERNAL 

REFERENCE 

In many cases where a broadband signal is corrupted by 
periodic interference there is no external reference input 
which is free of the signal. If a fixed delay is inserted 
in a reference input drawn directly from the primary input, 
as shown in Figure 2.9, the periodic interference can, in 
many cases, be cancelled. A key point is that the delay must 
be chosen to be of sufficient length to cause the broadband 
signal components in the reference input to become decorre- 
lated from those in the primary input. Because of their 
periodic nature, the interference components will remain 
correlated with each other. 

By taking the output from the LMS output instead of the 
difference signal the same system can be used to separate 
broadband interference from a periodic signal as shown in 
Figure 2.10. Such a filter is often called an adaptive 
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Fig. 2.9. Adaptive System for Cancelling Periodic 

Interference Without an External Reference 
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Fig. 2.10. Adaptive System for Cancelling Broadband 

Interference Without an External Reference 
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self-tuning filter. Additionally, applications of this filter 
are utilized in the adaptive line enhancer, a system used for 
detection of a low level signal imbeded in noise. The trans- 
fer function of this filter is the digital Fourier transform 
of the impulse response. Its magnitude at the frequency 
of the interference is very nearly one, the value required 
for perfect cancellation. 

H. THE ADAPTIVE NOISE CANCELLER AS A NOTCH FILTER 

One of the primary considerations for considering con- 
straints on adaptive filtering systems stems from the filter's 
use as a notch filter. After an analysis of the adaptive 
system as a notch filter it was felt that specific constraints 
could be used to 

a) shape the frequency characteristics 

b) produce a faster convergence. 

Before considering the constraining equations and their 
results it is first helpful to analyze the adaptive noise 
canceller in its notch filter mode. Figure 2.11 depicts a 
single frequency adaptive noise canceller with two adaptive 
weights. Analytical and experiemntal results show that if 
more than one frequency is present in the reference input 
then a notch for each will be formed. The primary input is 
assumed to be any type of signal — stochastic, deterministic, 
periodic, transient, or any combination of these. The 
reference input is assumed to be a pure cosine wave 
C cos (“gt + <j>) . The primary and reference inputs are sampled 
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Fig. 2.11. 



Single Frequency Adaptive Noise Canceller 




at the frequency of w s = 2(Pi)/T rad/sec. The reference 
input is sampled directly yielding XI (j), and after under- 
going a 90 degree phase shift, also produces X2(j). Assume 
synchronous sampling. 

A transfer function for the noise canceller of Figure 2.8-1 
can be obtained by analyzing signal propagation from the 
primary input to the system output. 

The weights are updated in accordance with the LMS 
algorithm yielding 



Wl(j + 1) = Wl(j) + 2y e ( j ) XI ( j ) 

W2 ( j + 1) = W2 ( j ) + 2ye(j)X2(j) 

The sampled reference inputs are 

XI (j) = C cos (oj o ( j ) T + <f>) 

X2(j) = C sin (ojq ( j ) T + <p) 



(2.43) 



(2.44) 



Using signal flow diagram techniques and considering that the 
error signal at time j = k is 

e ( j ) = 5(j-k) (2.45) 

the filter's impulse response at k = 0 can be given as 

y ( j ) = 2yC 2 u( j-1) cos (oj q ( j ) T) (2.46) 
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where u(j) is the discrete unit step function. The transfer 
function of this path is 

- z ( z - cosu.T) 

G ( z ) = 2yC z - 1 

z - 2 zcosuqT + 1 

2 

2yC ( z cos w n T - 1) 

= — 2 (2.47) 

z - 2z cos UqT + 1 

This transfer function can be expressed in terms of the 
radian sampling frequency = 2(Pi)/T as 

2 yC 2 [z cos (2-rrw u -1 ) -1] 

G ( z) = (2.48) 

z Z -2z cos (2iraj r .w ; + 1 
0 s 

When the feedback loop from the adaptive filter output to 
the difference signal is formed, the transfer function H(z) 
from primary input to noise canceller output can be written 
as 

2 -1 
z -2z cos (2 itu> w ) + 1 

H ( z ) = — 2 5 T (2.49) 

z -2(1 -yC )z cos ( 2 it oj n o) j +1 -2yC 

u s 

This transfer function has the property of a notch filter at 
the reference frequency to^. The zeros of the transfer func- 
tion are located in the z-plane at 

z = exp ( ± j2TTC0gaj s ^) (2.50) 

and are exactly on the unit circle at angles of ±2 ttu)qW s ^ 
radians. The poles are inside the unit circle at a radial 
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2 1/2 

distance (l-2uC ) ' from the origin. For slow adaptation 

2 

(small values of uC ) the angles of the poles are almost 

identical to the zeros. Since the zeros lie on the unit 

circle, the depth of the notch in the transfer function is 

infinite at frequency a> = u>q. The sharpness of the notch is 

determined by the closeness of the poles to the zeros. Corres 

ponding poles and zeros are separated by a distance approxi- 

2 

mately equal to pC . The notch bandwidth is given by 

BW = uC 2 co /n (2.51) 

s 

and the Q of the notch is determined by the ratio of the 
center frequency to the bandwidth. 

“0 

Q = —£■ (2.52) 

MC Ug 

The single frequency noise canceller is, therefore, equiva 
lent to a stable notch filter when the reference is a pure 
cosine wave. The depth of the null is generally superior 
to that of a fixed digital or analog filter because the 
adaptive process maintains the null exactly at the reference 
frequency. 
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III. THE CONSTRAINED ADAPTIVE FIR FILTER 



In this chapter the concept of the constrained adaptive 
filter is introduced. A literature search has indicated that 
very little research has been done in the area of constrained 
adaptive filters. Frost [Ref. 8] presents a constrained LMS 
algorithm which is capable of adjusting an array of sensors 
in real time to respond to a signal coming from a desired 
direction while discriminating against noises coming from 
other directions. A set of linear constraints on the weights 
maintains a chosen frequency characteristic for the array in 
the direction of interest. 

In this chapter three constraint conditions are presented. 
The first two involve a constraint on the angle between zero 
locations, so that this angle remains constant while the zero 
location changes adaptively. The constraint essentially 
maintains the pattern of the zeros while their location is 
shifted. The first approach is a direct implementation in 
which one of the weights is changed adaptively using the LMS 
algorithm and the others are slaved by the fixed angle formula 
to the adaptively adjusted weight. The second implementation 
involves a cascaded version of the foregoing master-slave 
concept. The third constraint considered is a linear con- 
straint on the weights. The approach presented uses a La- 
grangian multiplier formulation to augment the cost function 
in which the basic LMS adaptive algorithm is applied. 
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A. FIXED ANGLE CONSTRAINT — DIRECT IMPLEMENTATION 



Consider a transversal filter using the LMS algorithm 
with the constraint that the angular separation between the 
filter zeros in the z-plane is to be a constant as set by 
design requirements. As an example, consider a fourth order 
FIR filter with B = 0 2 -9^, where 9^ and 9 2 are the angles 
of the zeros. The value of B and the magnitude of the zeros 
is to be kept constant while the angles 9^ and 0 2 are to be 
adjusted adaptively. This specification serves to maintain 
the shape of the filter characteristics in the frequency 
domain. From the direct implementation of the transversal 



filter shown 


in Figure 3.1 the 


system transfer 


function is 


given by 








, -1 


, -i -2 


-3 -4 




H ( Z X ) = 


1 + W x z x + W 2 z 


+ W 3 z + W 4 z 


(3.1) 


= 


-4 4 3 

(z ) (z + W z + 


W 2 z 2 + W 3 z + W 4 ) 


(3.2) 



which can be factored into the form 



H ( z ) = 



_ N ( z ) _ 



2 2 2 
(z - 2r 1 cos9 ^z + r^) (z -2r 2 cos0 2 z 



+ r 2> 



D (z) 



(3.3) 



where r^ and r 2 represent the radius of the zero placement 
in the z-plane and 9^ and 9 2 are their respective angular 
displacement from the axis. Carrying out the indicated 
multiplication of the numerator of (3.3) yields 
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Fig. 3.1. 



Transversal Adaptive Filter — Direct 
Implementation 
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4 3 2 2 2 

N(z) = z - (2r^cos6^+2r 2 cos9 2 ) z + (r^+4r^r 2 cos9^cos0 2 +r 2 ) z 

- (2r^r 2 cos0 2 + 2r 2 r 1 cos0 1 ) z + r^r 2 (3.4) 

Setting these terms equal to the weight values of (3.2) 
yields 



II 

o 

S 


i 


(3.5) 


h-* 

II 


-2 (r^ cos 0^ + r 2 cos 0 2 ) 


(3.6) 


w 2 = 


2 2 
(r^ + 4r.. r 2 cos 0^ cos 0 2 + r 2 ) 


(3.7) 


22 

(jO 

II 


(-2r^r 2 ) (r^ cos 0 2 + r 2 cos 0^) 


(3.8) 




2 2 


(3.9) 


II 

S 


r l r 2 



The adaptive LMS algorithm must now be constrained so that 
the angle B is given by 

B = e - 0 2 (3.10) 

For simplicity let 

r^ = r 2 = R (3.11) 

From (3.10) it follows that 

9 1 (k+l) = B + 0 2 (k+l) (3.12) 

Using (3.11) and (3.12) Equation (3.6) can now be written as 
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w 1 (k+l) 



(3.13) 



= -2R (cos (B + 0^ (k+1) ) + cos(02(k+l)) 



Using the trigonometric identity 



cos (A+B) = cos A cos B - sin A sin B 



Equation (3.13) can be written as 

W^(k+1) = -2R (cos B cos ( 0^ (k+1) ) - sin B sin ( 0 2 (k+1) ) 

+ cos (0 (k+1) ) (3.14) 

Combining terms, and defining the constants 

K3 = -2R (cos (B) + 1) (3.15) 

K4 = -2R (sin (B) ) (3.16) 

Equation (3.13) can finally be written as 

W 1 (k+1) = K3 cos (0 (k+1) ) - K4 sin (0 2 (k+1) ) (3.17) 

This transcendental equation can be solved iteratively for 
0 2 (k+1) using the value for W^(k+1) obtained from the LMS 
algorithm 

W 1 (k+1) = W (k) + 2pX(k-l)e(k) (3.18) 

Now that © 2 (k+1) is known, Equation (3.7) can be solved for 
V ?2 (k+1) , where 
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w 2 (k+1) 



2 2 

= 2R + 4R cos (0 (k+1) + B) cos (0 (k+1)) (3.19) 

Recognizing that Equation (3.8) is proportional to Equation 
(3.6), the fourth algorithm iteration can be written as 

W 3 (k+1) = R 2 W 1 (k+l) (3.20) 

From (3.9) is a constant value 

W 4 (k+1) = R 4 (3.21) 

The four equations (3.18) through (3.21) comprise an 

adaptive iterative algorithm that produces the direct analyti- 
cal realization of Figure 3.1. It is seen that is changed 
adaptively and W 2 and are slaved to W^, and Wq and W 4 are 
constants. This approach yields two pairs of zeros which are 
located at radius R from the origin of the z plane and precisely 
B degrees apart. Figure 3.2 depicts the resulting z-plane 
diagram. 

B. FIXED ANGLE CONSTRAINT — CASCADED IMPLEMENTATION 

A cascaded implementation of the fourth order transversal 
filter is shown in Figure 3.3. The transfer functions result- 
ing from Figure 3.3 are given by 

2 2 
z - 2r. cos 0 . z + r . 

H 1 (z) = ^ (3.22) 

z 

for the first section, and for the second section 
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Fig. 3.2. Z-Plane Diagram of Zeros Resulting from 
Transversal Filter 

IV indicates four poles located at the 
Origin 
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Fig. 3.3. Cascaded Transversal Filter with LMS 

Adaptation of W^. Master-Slave Concept 



67 



(3.23) 



H 2 (z) 



2 2 
z - 2r 2 cos9 2 z + r 2 



The weight values from these two equations are 



W, 



-2 r^ cos 9^ 



= r 



W. 



-2 r 2 cos 0 2 



W 4 = r 2 

Once again the algorithm producing a set of converging 
weights begins with the LMS adaptive equation 



W^k+1) = W L (k) + 2pX(k-l) e (k) 



(3.24) 



The second weight is a constant proportional to 



W 2 (k+1) = R 



(3.25) 



From (3.22) the value for can be written as 



W. (k+1) = - 2 r^cos 0^(k+l) 



(3.26) 



Solving for 0^(k+l) 



cos9^ (k+1) 



-W 3 (k+1) 

2r~, 



(3.27) 



where the value of W^(k+1) is known from (3.24) and 
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R 



r 



1 



r 



2 



Therefore, 

-W 1 (k+1) 

0 1 (k+l) = Arccosine( ^ ) (3.28) 

Now the third weight can be written in the form of 

W^Ck+l) = -2 R cos02 (k+1) (3.29) 



where 



© 2 (k+1) = B - 0 x (k+1) (3.30) 

with B a given constant set by the user and 0^(k+l) known 
from (3.27). The fourth weight is proportional to 

W 4 (k+1) = R 2 (3.31) 

The four equations (3.24), (3.25) and (3.29), and (3.31) 

comprise an adaptive iterative algorithm that produces the 
cascaded realization of Figure 3.3. As in the direct reali- 
zation, the solution of Equations (3.22) and (3.23) using the 
adaptive weight solutions produces two pairs of zeros which 
are located at radius r^ = ^ = R from the origin of the 
z-plane and precisely B degrees apart. 

C. LINEARLY CONSTRAINED WEIGHTS— LAGRANGE MULTIPLIER APPROACH 
Consider now the LMS adaptive equations modified for a 
different type of constraint, namely a linear constraint on 
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the sum of the weight vector. That is, the constraint that 



W, (k) + W_ (k) + ... + W (k) = K (3.22) 

i z n 

where K is some user defined constant. This constraint func- 
tion is adjoined to the square of the error equation by the 
method of Lagrange multipliers [Ref. 9] . Recalling from 
Equation (1.5) that 

e (k) = d(k) - W(k) T X(k) 

the adjoined and modified least mean squared error function 
can be written as the cost function equation 

J (k) = [d (k) -W (k) T X (k) ] 2 - X (k) [W (k) T u - K] (3.33) 

where 



u = [1 1 ... 1] T 

and X (k) is the scalar Lagrange multiplier. 

In order to minimize the mean squared error under the 
given constraint consider the gradient of (3.33) with respect 
to the weights and to the Lagrangian multipliers. 

v[J(k)] = -2 [d(k) -W T (k)X(k) ]X(k) - X(k)u (3.34) 

and 
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-[W(k) T u - K] 



(3.35) 



V[J(k)) x = 

The method of steepest descent can be described by the two 
relationships 

W(k+1) = W(k) +k x v[j(k)] w (3.36) 

and 

A (k+1) = X (k) + k 2 V [ J (k) ] (3.37) 

Substituing Equations (3.34) and (3.35) into Equations (3.36) 
and (3.37) yields the final weight and Lagrange parameter 
iteration algorithms using the LMS approximation 

W (k+1) = w(k) - 2k 1 e(k)X(k) - X(k)u (3.38) 



and 

X (k+1) = X (k) - k 2 [W(k) T u - K] (3.39) 

where 

W(k) = the weight vector before adjustment 
W(k+1) = the weight vector after adjustment 

K = the linear constraint value 
k^,k 2 = the scalar constants (k^ f k 2 < 0) 
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A (k) 



the Lagrange parameter before adjustment 



\ (k+1) = the Lagrange parameter after adjustment. 

The resulting constrained adaptive filter is shown in Figure 
3.4. Experimental results are presented in the next chapter. 
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Fig. 3.4. The Adaptive LMS Filter with Linear Constraint 
and Lagrange Parameters 
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IV. SIMULATIONS AND RESULTS 



Using the Hewlett-Packard 85 (HP-85) microcomputer 
several computer simulations are performed to demonstrate 
both the unconstrained and the constrained LMS adaptive 
systems discussed in Chapters II and III. The unconstrained 
adaptive system utilized for computer simulation is the 
system depicted in Figure 2.1. The unconstrained results 
are obtained first with nine adaptive weights and then with 
only four adaptive weights. The noise input consists of 
either the sum of two sinewaves or zero mean white noise 
added to a sinewave. In all unconstrained cases the 
reference input (the desired waveform) is a sinewave. The 
constrained adaptive systems are simulated for the linear 
constraint using the Lagrangian multiplier and for the 
angular constraint (separation of zeros in the z-plane 
plot) in both cascaded and direct implementations. 

A. THE UNCONSTRAINED ADAPTIVE NOISE CANCELLER 

The adaptive noise canceller shown in Figure 2.1 is 
computer simulated using nine weights, each adapted by the 
LMS algorithm. The desired signal is a 10 Hz sinewave 
sampled at f = 128 Hz. The noise input is the sum of two 
equal sinewaves with frequency f = 10 Hz and frequency 
f = 35 Hz. The number of samples is N = 128 and the 
adaptation constant is u = K = 0.1. The system output 
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converges after a short learning period to the desired 
signal. Figures 4.1 through 4.3 depict these results. 

A sample of three of the nine weights (w^ , w^ , Wg) is 
shown in Figures 4.4 through 4.6 to illustrate the steady 
state solution of the adaptive weights. After just N = 24 
all weights have settled to within 0.1% of their final 
value. Figure 4.7 is the system error. It should be noted 
that a faster sampling frequency of f = 256 reduces this 
error even further. 

The next simulation is similar to the first with the 
exception that this system uses only four weights instead 
of the nine weights used previously. Again the noise input 
is the sum of two equal sinewaves (f = 4 Hz, f = 20 Hz) and 

the reference or desired waveform is a sinewave at f = 4 Hz. 

The adaptation constant is K1 = 0.1, N = 128 and the 

sampling frequency for the inputs is f = 128 Hz. Figures 4.8 

and 4.9 show the filter input and the reference signal. It 
is evident from the system output shown in Figure 4.10 that 
this system does not track as well as the previous system 
with nine weights. Plotting weight #1 of the system through 
time, as shown in Figure 4.11, illustrates the oscillatory 
nature of the weight values. 

The same unconstrained simulation is repeated with the 
noise input taken as zero mean uncorrelated noise (generated 
by the HP-85 random number generator) summed with a sinewave 
of frequency f = 10 Hz. Figure 4.12 shows the noise input. 
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Fig. 4.1. System Input - Nine Adaptive Non- 
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^*^■9* 4.2. System Reference Signal 

Sampling Frequency F = 128 Hz 
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Fig. 



FILTER OUTPUT 




4.3. System Output - Nine Adaptive Weights 
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Fig. 4.4. Weight #0 Plotted Through N = 128 Samples 
Nine Adaptive Weights 
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Fig. 4.5. Weight #4 Plotted Through N = 128 Samples 
Nine Adaptive Weights 
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Fig. 4.6. Weight #8 Plotted Through N = 128 Samples 
Nine Adaptive Weights • 
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Fig. 4.7. System Error - Nine Adaptive Weights 
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4.8. System Noise Input - Four Adaptive Weights 



83 



MflG 



DESIRED SIGNAL 




Fig. 4.9. System Reference 
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Fig. 
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4.10. System Output - Four Adaptive Weights 
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Fig. 4.11. Weight #1 Plotted Through N = 128 Samples 
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System Input - S/M 
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The desired signal is the sinewave shown in Figure 4.13. 

Nine adaptive weights are utilized. The system sampling 
frequency is f = 256 Hz and the adaptation constant is 
K1 = 0.1. The signal-to-noise ratio of the input signal 
is 1.64 dB. Figure 4.14 shows the system output while 
Figure 4.15 illustrates the system error. Figure 4.16 shows 
one of the system weights plotted through time. 

B. CONSTRAINED ADAPTIVE SYSTEM SIMULATIONS 

The simulation of the angularly constrained adaptive 
systems shown in Figures 3.1 and 3.3 produces nearly identical 
results. Computationally, the iterative solution required 
for the direct implementation is much slower than that of 
the cascaded system. Since both system outputs are virtually 
identical, those of the faster cascaded system are shown. 

All of the linearly constrained adaptive results utilize 
nine adaptive weights and all of the angularly constrained 
systems utilize four weights. 

Using the cascaded adaptive system shown in Figure 3.3 
simulation results are obtained for a noise input shown in 
Figure 4.17 of two sinewaves of frequencies f = 4 Hz and 
f = 20 Hz. The sampling frequency f = 128 Hz. The 
desired signal shown in Figure 4.18 is a sinewave of 
frequency f = 4Hz. The adaptation constant is Kl = 0.1. 

The zeros are desired at a radius of 1 and with a separation 
of 12 degrees. The system output tracks the desired signal 
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Fig. 4.13. System Reference Signal 
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Fig. 4.14. System Output - Four Adaptive Weights 
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Fig. 4.15. System Error - Four Adaptive Weights 
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Fig. 4.16. Weight #1 Plotted Through N = 128 Samples 
Four Adaptive Weights 
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Fig. 4.18. Reference Input 
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as shown in Figure 4.19 and places the system zeros with 
the exact requested degree of separation as shown in 
Figure 4.20. Zero 1 is at radius 1 and an angle of 63.39 
degrees. Zero 2 is at a radius of 1 and an angle of 51.39 
degrees. The other two zeros are the conjugate pairs of 
the first two. 

The adapted weight number 1 is plotted through time in 
Figure 4.21. The same oscillating pattern is shown here as 
is depicted in Figure 4.11. It is felt that, as in the case 
of Figure 4.11, using a greater number of weights would 
eliminate the noisy weight pattern and accordingly produce 
a cleaner output. 

Suppose now that the user wishes to drastically alter 
the frequency characteristics. The input to the system 
remains the same (as shown in Figure 4.17) however now the 
desired angle of separation of the zeros is 130 degrees 
instead of the original 12 degrees. System output is shown 
in Figure 4.22, the reference signal is shown in Figure 4.23, 
and the z-plane zero location is shown in Figure 4.24. 

Zero number 1 is at a radius of 1 and at an angle of 158.55 
degrees and zero number 2 is at a radius of 1 and an angle 
of 28.55 degrees, providing the prescribed 130 degrees of 
separation. The remaining two zeros form the conjugate pair. 
The sampling frequency is changed to f = 256 Hz and the 
adaptation constant K1 is changed to 0.05. 
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Fig. 4.19. System Output - Four Adaptive Weights 
Desired Zero Separation = 12 Degrees 
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Fig. 4.20. Z-Plane Zero Location - Four Adaptive 
Weights 
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Fig. 4.21. Weight #1 Plotted Through N = 128 Samples 
Four Adaptive Weights 
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Fig. 4.24. Z-Plane Zero Location 



The same simulations are performed for two sinusoids of 
frequencies f = 10 Hz and f = 35 Hz. A sampling frequency 
of f = 256 Hz is used with an adaptation constant K1 = 0.1. 
Figures 4.25 through 4.28 depict the system input (the sum 
of the two sinusoids) , the filter output, the desired signal, 
and the z-plane zero locations for a desired separation of 
20 degrees. Zero 1 is located at a radius of 1 and an angle 
of 50.2 degrees. Zero 2 is located at a radius of 1 and an 
angle of 70.2 degrees. The remaining two zeros are the 
conjugate pairs. Figures 4.29 and 4.30 show the system out- 
put and z-plane zero location for the same system input and 
same reference but with a desired zero separation of 40 
degrees. Zero 1 is located at a radius of 1 and an angle 
of 45.43 degrees and zero 2 is located at a radius of 1 and 
an angle of 85.43 degrees. The remaining two zeros are 
conjugate pairs. 

C. LINEARLY CONSTRAINED ADAPTIVE FILTER SIMULATIONS 

USING THE LAGRANGE MULTIPLIER TECHNIQUE 

The linearly constrained LMS adaptive filter shown in 
Figure 3.4 is simulated for various values of K, the linear 
constraint value. Two types of system inputs are used — 
random "white" noise and a sinusoidal signal of frequency 
4 Hz. In each case the desired signal is a damped exponen- 
tial with a final value of 0.2. 

Figure 4.31 depicts the filter output superimposed upon 
the desired output of a damped exponential of final value 
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Desired Zero Separation =20 Degrees 



104 



own 



DESIRED FILTER OUT 




Fig. 4.27. Reference Signal 
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Fig. 4.28. Z-Plane Zero Locations - Four Adaptive 
Weights 
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Fig. 4.29.- System Output - Four Adaptive Weights 
Desired Zero Separation = 40 Degrees 
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Fig. 4.30. Z-Plane Zero Location 
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4.31. System Output and Desired Signal 
Linear Constraint K = 0 
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0.2. The linear constraint K is arbitrarily chosen to be 
zero. The adaptation constant is K1 = 1 E-6. The system 
error is shown in Figure 4.32. The sum of the weights plotted 
through time is shown in Figure 4.33. The final value of 
the Lagrange multiplier, that is A (N) at time N = 64, is 
found to be -1.022 E-4. The final value of the sum of 
weights is found to be 0.059 compared with a requested value 
of zero. 

The same system is again simulated for a linear con- 
straint of K = -0.2. Figure 4.34 shows that the sinusoidal 
input converges to the desired damped exponential of final 
value 0.2. Interestingly, the response converges and then 
begins to build an oscillation. This type of response in 
adaptive systems has been observed elsewhere and may be due 
to an arithmetic precision problem [Ref. 10]. Figure 4.35 
shows the system error and Figure 4.36 shows the sum of 
weights graph plotted through time. The final value of the 
sum of weights is found to be -0.2427 compared with the 
desired value of -0.2. 

The simulation is again performed, now with a "white" 
noise input and the same damped exponential with final value 
of 0.2 as the desired signal. In the first of the three 
noise input simulations the sum of weights constraint K is 
set to -0.1; the second simulation has K = 0.2; the final 
simulation uses K = 0.5. Figures 4.37 through 4.42 depict 
the simulation results. In Figure 4.42 the sum of the 
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Fig. 4.32. System Error y(n) 



111 



dhw 



I N P U T = S I N U S 0 I D 
DESIRED SIG=OfiMPED EXP 









1_1 — j — t 


|— H ! >■ — H-H 

N = 6 4 


SAMPLES 


.» 








SUM OF 


WEIGHTS 



Fig. 4.33. Sum of Weights Plotted Through N = 128 
Samples. Constraint K = 0 
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Fig. 4.34. System Output and Desired Signal 
K = -.2 
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Fig. 4.35. System Error 

Linear Constraint K = -.2 
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Fig. 4.36. Sum of Weights Plotted Through N = 128 
Samples. Linear Constraint K = -.2 
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Fig. 4.37. System Output Superimposed over 

Desired Signal of Damped Exponential 
K = ~.l. Nine Adaptive Weights 
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Fig. 4.38. Sum of Weights Plotted Through N = 64 
Samples. Constraint K = -0.1 
Nine Adaptive Weights 
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Fig. 4.39. System Output Superimposed over Desired 
Signal of Damped Exponential K = +0.2 
Nine Adaptive Weights 
Kl = .00001 K2 = .00008 
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Fig. 4.40. Sum of Weights Plotted for N = 128 Samples 
Nine Adaptive Weights. K = .2 
K1 = .00001 K2 » .00008 
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Fig. 4.41. System Output Superimposed over the 

Desired Signal of a Damped Exponential 
K = .5 

Kl = .00001 K2 = .00009 
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Fig. 4.42. Sum of Weights Plotted Through N = 128 
Samples . K = . 5 
Kl = .00001 K2 = .00009 
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weights overshoots the constraint of K = 0.5 but returns 
to within 10% of that value at N = 128. 

D. CONCLUSIONS 

This thesis has investigated adaptive noise cancellation 
techniques in detail with the objective of implementing 
constrained adaptive filters. Two basic types of constrained 
adaptive filters are presented — the fixed angular separation 
(direct and cascaded implementations) and the adaptive 
Lagrangian multiplier linear constraint approach. All three 
methods produce the desired output within a very close 
tolerance for the examples tested. As expected, the direct 
implementation takes longer to converge than the cascade 
approach. The results are promising, however, several key 
questions remain to be investigated. In particular the 
convergence properties of these constrained adaptive 
algorithms need to be studied in detail. In the case of 
the Lagrangian multiplier approach, if there is an optimal 
steady-state solution, the modified LMS algorithm should 
find a minimum since it is a simplified gradient technique. 
However the LMS minimum may not be global. For the case of 
the angular constraint process, the stability of the 
adaptive algorithm and its convergence also need to be 
examined, although the master- slave would seem to contain 
inherent stability. 
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Finally, the concepts of constrained adaptive algorithms 
need to be examined with more sophisticated examples using 
a larger number of weights. 
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